MagneTip: Reintroducing a Physical Interaction Loop for 3D Musical Drawing in Extended Reality

Extended Reality interfaces open numerous opportunities for musical expression.
One of them is 3D musical drawing, \ie the ability to draw sonic and visual paths in virtual and physical spaces.
However, in existing instruments (controllers, hand tracking) most of the interaction happens in the digital/virtual space, which reduces primary audio and haptic feedback and might limit the expressiveness one has with physical drawing and acoustic instruments.
In this paper, we propose a novel design approach which moves part of the 3D interaction back to the physical space for more intimate controls and more direct feedback, creating a physical interaction loop connected with the virtual interaction loop.
We then present MagneTip, a first implementation of this approach for 3D musical drawing, which enables one or two handed interaction and combines co-localised and spatialised feedback.